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Abstract 

Assume that a family of stochastic processes on some Polish space E converges to a de- 
terministic process; the convergence is in distribution (hence in probability) at every fixed 
point in time. This assumption holds for a large family of processes, among which many 
mean field interaction models and is weaker than previously assumed. We show that any 
limit point of an invariant probability of the stochastic process is an invariant probability 
of the deterministic process. The results are valid in discrete and in continuous time. 

1. Introduction 

This paper is motivated by results on mean field interaction models and stochastic ap- 
proximation algo rithms which obtain convergence of a family of stochastic processes to a de- 
term i nistic limit (lKurtzl.ll970l: ISandholml . l2006t iBordenave et al.l . 120071 : iGraham and Meleard 
19941 : iBenai'm and Le Bonded . l2008l ). Often, convergence is over finite time horizons, which 
asks the question of whether the convergence extends to the stationary regime. In this pa- 
per, we show that some form of convergence of the stationary regimes follows systematically 
from convergence at any fixed and finite time horizon, under very weak assumptions. Pre- 
vious answers to this question exist with stronger assumptions than here; for example , 
the s pace is finite dimensional and the deterministi c limit is a dissipative ODE ( iBenai'ml . 



19981 ) , or the set of invariant distributions is tight flFort and Pagesl. Il999[). Such assump- 



tions cannot always be made, consider for example the cases in fIChaintreau et al.l . 12009 



Bordenave et al.l . 120071 ): our result requires much weaker assumptions and appears to be 
more general. 

More precisely, we consider a family of stochastic processes , indexed by A = 1, 2, ... 
over some Polish space E\ we assume that the processes have the property that, as A — >■ oo 
and the initial conditions |/^(0) — )■ |/(0), the marginals of the process Y^{t) converge in 
distribution to some deterministic y{t), where convergence is at every fixed time t (see 
Hypothesis [1] below) . We show that, under the (mild) assumption that the processes are 
Feller, this is sufficient to obtain that any limit point of an invariant probability of 
is an invariant probability of the deterministic process (and thus its support is included 
in its Birkhoff center). Note that we do not need to assume any semi-flow nor continuity 
property for the limiting deterministic process. 

In the special case where the deterministic process has a unique limit point y* = 
limt^oo y{t) and where the sequence of invariant probabilities 11^ is tight, it follows imme- 
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diately that 11^ converges to the Dirac mass at y*. This result is known in the context of 
stochastic approximation algorithms; our results here extend it to a more general setting. 

Our result is also more general as it applies to other cases. Mean field interaction models 
were often used as practical approximations of complex interacting object systems, where 
the stationary distribution of the system is approximated by the stationary regime 
of an ordinary differential equation (ODE); this was applied for example to TCP connec- 



tions ( iTinnakornsrisuphap and Makowskil . l2003l : iBaccelli et al.l . l2006l : I Graham and Robert 



20091 ). H TTP flows (jBaccelli et al.l. 120041) . bandwidth sha ring between streaming and flle 



transfers (jKumar and Massoulie. 120071) . mobile networks fIChaintreau et al.l. 12009 ). robo t 



swarms 



f Martinoli and Eastonl. 2002[) , tran sportation systems f Afanassieva et al. 



19971 ). 



reputation systems (ILe Boudec et al.l . 120071 ). just to name a few. Previous results are ob- 
tained when the ODE has a unique limit point to which all trajectories converge. Not only 
does our result extends this flnding to more general spa ces, it also extends it t o the cases 
where there is not a unique limit point. For example, in ( iBordenave et al.l . 120071 ). the ODE 
a unique limit point under some restrictive assumptions on the model p arameters; i f thes e 
assumptions do not hold, the ODE r aay have limit cycles, a s shown in (ICho et al.l . |2010| ). 
and nothing can be concluded from (IBordenave et al.l . 120071 ). Using our results, it follows 
that the limit points of any invariant probability has a support included in the limit cycles. 



2. Assumptions and Notation 

2.1. A Collection of Random Processes 

Let {E, d) be a Polish space and V{E) the set of probability measures on E, endowed 
with the topology of weak convergence. Let Cb{E) be the set of bounded continuous 
functions from E to M. 

We are given a collection of probability spaces {^l^ , , P^) indexed by = 1, 2, 3, ... 
and for every N we have a process deflned on (l)^, J-"^, P^). Time is either discrete 
or continuous. In the discrete time case, y^(t) is a collection of random variables indexed 
by A = 1, 2, 3... and t E N. In the continuous time case, let De[0, oo) be the set of cadlag 
functions [0, oo) — )■ E; Y'^ is then a stochastic process with sample paths in De[0, oo). 

We denote by y^(t) the random value of Y^ at time t > 0. Let E'^ C E he the 
support of F^(0), so that P^(y^(0) e E^) = 1. 

We assume that, for every A^, the process Y'^ is Feller, in the sense that for every t > 
and h G Cb(E), [h{Y^ {t))\Y^ (0) = yo] is a continuous function of yo G E. 

Definition 1. A probability 11^ G V{E) is invariant for Y'^ if Il^{E^) = 1 and for every 
h G Cb{E) and every t > 0: 

[ [/z(F^(t))|F^(0)=2/]n^(dy)= / %)n^(rf2/) 
Je J e 
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2.2. A Deterministic Measurable Process 

Further, let </? be a deterministic process, i.e. a mapping 

ip: Tx E E 

where T = N or T = [0, oo). 

We assume that Lpt is measurable for every fixed time t > 0. Note that there is no 
assumption here that Lp is continuous nor that it is a fiow. 

Definition 2. A probability 11 G V{E) is invariant for ip if for every h G Cb{E) and every 
t > 0: 

/ hiiptiy))n{dy)= [ h{y)U{dy) 

J E J E 

2.3. Convergence Hypothesis 

We assume that, for every fixed t the processes converge in distribution to the 
deterministic process (/9 as — )■ oo for every collection of converging initial conditions. 
More precisely: 

Hypothesis 1. For every y^ in E, every sequence {yQ)N=i.2,... such that yQ G E^ and 
limAr_j.oo = Vo, o.nd every t > 0, the conditional law ofY^[t) given y^(0) = y^ 
converges weakly to the Dirac mass at Lpt{yo). That is 

hm y^(0) = y^] = h o ^,(yo) 

for all h G Cb{E) and any fixed t >0. 

2.4. Examples. 
In discrete time, is a Markov chain on E^ , 

E = V{S) for some compact set S, Y^ is the occupancy measure of a process on 5", and E^ 
is the set of probabilities that are the sum of N Dirac masses. Here Definition [1] coincides 
with invariant probability for a Markov chain. The deterministic process is an iterated 

map, and Definition [2] coincides with invariant probability of an iterated map. 

In continuous time, Y^ rnay be a Markov process on E^ . as in (IKurta. ll970l:ISandholrnl . 



as in (ILe Boudec et al.1 . 120071 ) , where 



20061 : iBordenave et al.l . 120071 : iGraham and Meleardl . 1 19941 : iBenai'm and Le Bonded . l2008l ) 



Definition [T] coincides here with invariant probability for a Markov process. The deter- 
ministic process is a semi-fiow, and Definition [2] coincides with invariant probability for 
semi-fiows. If E is finite dimensional, the deterministic process is an ODE or a differential 
inclusion. 

Still in continuo us time, Y^ may also be the continuous linear interpolat ion of a discrete 
time process, as in f Bordenave et al. . 2007; Benai'm and Le Boudec . 20081 ) (in this case it 
is not a Markov process). An invariant probability for Y^ is here an invariant probability 
of the interpolated Mark ov chain. 



H ypothesis [Dholds in flLe Boudec et al.l . 120071 : iKurtzl . Il970l : ISandholml . l2006l : IBordenave et al 



20071 : iBenai'm and Le Bouded . l2008l ) as a consequence of stronger convergence results; for 
example in flKurtzl . Il970l ) there is almost sure, uniform convergence for all t G [0,T], for 
any T > 0. 
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3. Convergence of Invariant Probabilities 

3.1. Main Theorem 

Theorem 1. Assume HypothesisUl holds and letU G 'P{E) he a limit point of the sequence 
li^ , where li^ is an invariant probability for . Then 11 is an invariant probability for 



Proof. Let be a subsequence such that limfc_5.oo n^*-' = 11 in the weak topolog y on V{E). 



By Skorohod's representation theorem for Polish spaces (lEthier and Kurtg . l2005l . Thm 1.8), 
there exists a common probability space J-", P) on which some random variables for 
/c G N and X are defined such that 

law of X^ = 
law of X = n 
X'' ^XF-a.s. 

Fix some t > and h G Cb{E), and define, for A; G N and y E E 

a''{y)'^E{h{Y^''{t))\Y^^{0)=y) 
Since 11^'= is invariant for y^*: 

a\y)U''''{dy)= [ h{y)U'''' {dy) (1) 

E Je 

Hypothesis [1] implies that Ymvk^^ {x^) = h{ipt{x)) for every sequence such that 
x^ G -E^* and lim/j^oo = x E E. Now X^ G E^'= P— almost surely, since the law of X^ 
is n^'' and II^* is invariant for Y^'^. Further, X^ — X P— almost surely; thus 

lim a^{X^) = h{ipt{X)) P- almost surely (2) 

k—>oo 

Now a'^(X^) < \\h\\^ and, thus, by dominated convergence: 

lim E (a^X^)) = E {h{^tiX))) (3) 



Using Eq.dD): 



and thus 



hm / h{y)X\^-{dy)= f hMy))Il{dy) (4) 
>^^°°Je Je 



h{y)Ii{dy)= / h{ipt{y))Ii{dy) (5) 

J E 

This holds for any h G Cb{E) and t > 0, which shows that 11 is invariant for ip. □ 
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3.2. The Continuous Semi-Flow Case 

Note that our assumptions on Lp are very weak. We now make an additional assumption: 



Definition 3. The deterministic process is a continuous semi-flow if 

1- <^o(2/) = y 

2. Lfs^t = '^s° 'ft for all nonnegative s and t 

3. i^t{y) is continuous in t and y 



If 9? is a continuous semi-flow, it follows from Poincare's recurrence theorem flMane and Levyi . 

19871 ) that the support of any limit point of 11^ is included in the closure of the recurrent 



set : 

= {x E E : liminf d{x, ft{x)) = 0}. 

t—^oo 

In particular, if the semi-flow has a unique limit point, we have: 

Corollary 1. Assume HypothesisUl holds and that is a continuous semi-flow. Let 
be a sequence of invariant probabilities for . Assume that 

1. the sequence (n^)Ar=i^2,... is tight; 

2. there is some y* E E such that for all y E E, limt_j.oo Vtiv) = V* ■ 

It follows that the sequence 11^ converges weakly to the Dirac mass at y* . 

Recall that tightness means that for every e > there is some compact set K (Z E such 
that Il^{K) > 1 — e for all A^. If E is compact then (n^)jv=i,2,... is automatically tight. 
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